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ABSTRACT 


An  improved  assessment  methodology  has  been  implemented  at  NOSC  and  tested  using  an 
instrumented  peg-in-hole  (PiH)  taskboard.  Several  aspects  of  the  methodology  are  discussed  in 
light  of  their  implications  for  future  studies  of  manipulator  performance.  Using  a  simple  (but  high- 
fidelity)force-feedback  manipulator,  a  group  of  9  trained  operators  showed  a  consistent  advantage 
for  stereoscopic  TV  viewing  over  monoscopic  TV  viewing  when  performing  the  PiH  task.  To 
introduce  a  controlled  element  of  spatial  uncertainty  into  the  testing  procedure,  taskboard 
orientation  relative  to  the  manipulator  and  remote  video  camera  head  was  changed  in  a 
randomized  order  on  a  trial-by-trial  basis.  The  stereoscopic  advantage  demonstrated  by  this  study 
can  reasonably  be  expected  to  be  even  more  pronounced  as  the  quality  of  the  stereo  TV  interface  is 
improved  and  force-feedback  provided  through  the  manipulator  system  is  diminished  and/or 
distorted. 


2.  INTRODUCTION 

2.1  Is  Stereoscopic  TV  Advantageous  for  Remote  Manipulation? 

The  controversy  continues  to  this  day,  seemingly  unabated.  Nearly  30  years  after  the 
earliest  published  laboratory  comparisons  of  remote  manipulation  under  monoscopic  (mono)  and 
stereoscopic  (stereo)  TV  viewing  conditions,  system  designers  in  a  variety  of  applications  areas  are 
still  scratching  their  heads  over  the  question  of  whether  the  added  expense  and  complexity  of 
stereo  TV  are  truly  worth  the  investment.  It  is  not  difficult  to  cite  several  reports  in  the  published 
literature  that  support  either  a  "pro  or  con"  stereo  TV  position.  Why  has  a  simple  yes  or  no  answer 
remained  so  elusive?  Because  the  question,  itself,  is  deceptively  simple  as  commonly  stated. 

The  success  or  failure  of  any  remote  manipulator  operation  is  dependent  on  three  basic 
aspects  of  the  work  situation:  1)  the  operator  ,  2)  the  equipment  (including  sensors,  displays, 
controllers,  effectors,  and  the  datalink)  ,  and  3)  the  task.  Human  variability  is  a  given.  Operators 
bring  different  levels  of  physical  and  mental  skills  to  bear  on  the  operation  of  any  manipulator 
system.  How  effectively  they  use  their  physical  and  mental  resources  is  strongly  conditioned  by 
their  previous  experiences  with  the  equipment  as  well  as  their  previous  experiences  with  the  task. 
Stereo  TV  is,  of  course,  one  possible  feature  of  the  equipment  and  there  are  a  great  many  ways  that 
stereo  TV  can  be  implemented,  to  varying  degrees  of  acceptability.  A  finding  of  no  performance 
difference  between  stereo  and  mono  TV  may  simply  be  a  matter  of  poor  implementation  of  the  stereo 
TV  equipment  or  a  matter  of  poor  comparability  between  the  stereo  and  mono  systems  tested  [1].  One 
might  reasonably  expect  that  as  overall  quality  of  the  equipment  improves,  that  the  performance 
gained  by  improving  one  feature  of  the  equipment  would  diminish.  For  instance,  the  stereo  TV 
performance  advantage  is  likely  to  be  less  pronounced  (but  not  necessarily  less  reliable)  with  a 
high-fidelity  force-feedback  manipulator  than  with  one  providing  no  force-feedback.  Similarly, 
as  the  operators'  familiarity  with  the  equipment  and  the  task  improves,  the  performance  gain 
derived  from  improving  a  single  feature  is  likely  to  be  less  pronounced.  Lastly,  the  particulars  of 
the  task  to  be  undertaken  and  its  environmental  context  have  a  direct  and  obvious  impact  on  how 
well  it  can  be  done.  The  task  domain  for  remote  manipulation  is  ever-growing.  Demands  for  ma- 
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equivalent  manual  dexterity  from  remotely  operated  manipulators  have  been  on  the  rise  given  the 
phenomenal  success  of  computing  technologies  and  their  adaptation  into  remote  manipulation 
systems.  Under  most  real-world  conditions  where  environmental  visibility  is  adequate, 
manipulative  capability  is  very  directly  dependent  on  the  ability  to  correctly  perceive  the  spatial 
relationships  between  the  manipulator  and  the  objects  to  be  manipulated.  To  the  extent  that  stereo 
TV  provides  the  operator  with  a  more  accurate  perception  of  the  relative  depths  of  objects  as  well 
as  their  3-dimensional  shapes,  one  can  expect  it  to  produce  performance  advantages. 

A  fair,  unbiased  approach  to  comparing  various  features  of  remote  manipulation  systems 
must  take  into  account  and  experimentally  control  for  influences  on  performance  outcomes  arising 
from  all  3  of  the  major  areas  of  influence  outlined  above.  Such  a  methodology  must  be  founded  on 
competent  engineering  practices  for  design  and  maintenance  of  equipment  as  well  as  a  close 
adherence  to  standards  for  conducting  behavioral  research  [2]  and  statistical  analysis  [3]  practiced 
by  the  social  science  research  community. 

2.2  Improvements  in  Testing  Methods  at  NOSC 

During  the  past  year  at  NOSC-Hawaii,  we  have  improved  testing  methods  for  comparing 
various  features  of  remote  manipulation  systems.  This  has  involved  implementing  more  rigorous 
procedures  for  operator  screening  and  taking  greater  precautions  to  isolate  operators  from  extraneous 
sources  of  stimulation,  both  visual  and  auditory,  during  test  sessions.  Data  collection  is  now  fully 
automated,  providing  better  standardization  of  procedures  as  well  as  more  precise  and  completely 
objective  measurements  of  operator  responses.  Full  automation  of  the  test  procedure  also  makes  it 
convenient  to  completely  randomize  orders  of  presentation  of  the  various  conditions  or  system 
features  to  be  compared.  It  supports  the  utilization  of  within-subjects  test  designs  while  controlling 
for  the  potential  contamination  of  carry-over  effects.  3y  precisely  controlling  the  position  and 
orientation  of  a  taskboard  relative  to  the  sensors  and  the  manipulator  arm,  it  also  allows  us  to 
introduce  and  control  the  effects  of  positional  and  orientational  uncertainty  that,  we  believe,  exert 
a  substantial  effect  on  remote  manipulation  under  real-world  operating  conditions  [61. 

Given  these  improvements  to  the  laboratory  facility  and  testing  procedures,  we  collected 
performance  data  which  addresses  the  question  of  the  stereo  TV  advantage  for  remote 
manipulation  using  a  pcg-in-hole  (PiH)  task.  The  methods,  results,  and  findings  of  a  recent 
experiment  are  summarized  in  the  remainder  of  this  report. 

2.3  Experimental  Issues  Tested 


The  single  experiment  described  herein  addressed  several  issues  that  bear  on  the  usefulness 
of  stereo  displays  as  well  as  unique  features  of  the  task  which  was  selected  to  show  performance 
differences.  In  the  process  of  training  operators,  we  were  interested  in  measuring  the  amount  of 
practice  with  the  manipulator,  PiH  task,  and  viewing  system  that  was  necessary  before  learning 
effects  "levelled  off"  in  the  peg  movement  time  data  .  Next,  by  having  operators  use  pegs  of  2 
different  diameters,  we  included  a  variation  in  task  difficulty  that  has  consistently  shown  large 
differences  in  previous  studies  [7,8].  This  factor  was  included  as  a  validation  of  the  overall 
screening,  training,  and  testing  procedures.  We  hypothesized  that  movement  times  would  be  slower 
for  tighter  peg-hole  tolerances  in  general  accord  with  Fitt's  law.  In  addition,  we  hypothesized 
that  stereo  TV  viewing  would  provide  a  performance  advantage  over  mono  TV  viewing.  This 
finding  would  essentially  replicate  and  therefore  support  the  findings  of  a  previous  study  |9)  with 
the  same  manipulator  and  a  similar  task  using  "highly-practiced  operators'.Bv  introducing  and 
experimentally  controlling  for  orientational  uncertainty  of  the  taskboard  on  a  trial-bv-trial  basis, 
we  hypothesized  that  the  stereo  advantage  would  be  even  more  pronounced  when  the  taskboard 
was  re-oriented  on  a  trial-by-trial  basis  than  it  would  when  it  remained  fixed  in  orientation  over  a 
large  block  of  trials.  By  choosing  relatively  small  deviations  of  the  taskboard  from  a  "normal" 
position,  we  hoped  to  show  the  potency  of  the  interaction  between  stereo  viewing  and  orientational 
uncertainty.  Finally,  we  were  interested  in  measuring  the  relative  efficiencies  of  remote 
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manipulation  under  direct  view/remote  manipulation  and  direct  view/direct  manipulation  control 
conditions. 


3.  EXPERIMENTAL  METHODS 


3.1  Operators 

Eight  male  operators  and  one  female  operator  participated  on  a  voluntary  basis.  Their  ages 
ranged  from  21  to  47  years.  Two  of  the  operators  had  extensive  prior  experience  using  the  CRL 
Model-G  manipulator,  two  had  some  limited  experience  with  another  manipulator,  and  five  had 
never  operated  a  remote  manipulator  prior  to  the  experiment.  All  operators  were  screened  for 
normal  visual  acuity  in  both  eyes  ,  vertical  and  lateral  phorias,  and  the  ability  to  fuse  stereo 
images  and  perceive  forms  in  depth  using  a  standard  Armed  Forces  Vision  Tester  and  random  dot 
stereogram  pairs.  One  of  the  male  operators  was  left  handed.  All  operators  used  their  right  arms 
and  hands  to  control  the  manipulator.  To  deny  operators  the  use  of  auditory  cues  produced  by  the 
manipulator,  contact  of  the  peg  with  the  taskboard,  or  the  robot  arm  which  moved  the  taskboard, 
operators  inserted  plugs  (E.A.R.)  in  their  ears  and  donned  a  pair  of  headphones  through  which 
"pink  noise"  was  played  at  sufficient  volume  to  mask  external  noise  sources.  This  arrangement  had 
the  additional  virtue  of  isolating  the  operators  from  all  forms  of  auditory  distractions  in  the  lab 
facility  (phones  ringing,  conversations,  military  aircraft  noises).  Operators  were  informed  of  the 
purpose  of  the  experiment  and  were  instructed  to  place  greatest  emphasis  on  avoiding  errors  such  as 
inadvertent  collisions  or  use  of  excessive  force,  but  to  move  the  peg  as  quickly  as  possible. 

3.2  Equipment 

A  pair  of  black  and  white  CCD  video  cameras  (Pulnix  Model  TM-540)  with  8mm  lenses 
were  used.  Each  provided  coverage  over  a  measured  horizontal  field  of  view  of  53  degrees.  Each 
camera  was  attached  to  an  adjustable  3-axis  (i.e,  pan,  tilt,  and  roll)  camera  mount  which  permitted 
a  controlled  calibration  of  the  camera  pair  prior  to  testing.  The  cameras  and  their  mounts  were 
attached  to  a  heavy-duty  tripod,  and  their  position  remained  fixed  throughout  all  practice  and 
experimental  sessions.  Cameras  were  converged  to  and  focussed  on  the  center  point  of  the  PiH 
taskboard  which  was  1.2  meters  distant.  Cameras  aimed  down  to  the  taskboard  center  with  a 
lookdown  angle  from  horizontal  of  10°.  Interaxial  separation  between  the  converged  cameras  was 
fixed  at  6.5  cm. 

A  dual-monitor  beamsplitter  display  with  linear  polarizer  filters  (Polaroid  HN38)  for 
channel  separation  was  used  for  the  operator's  TV  view  of  the  task.  For  those  unfamiliar  with  the 
general  configuration  of  a  beamsplitter  stereo  display,  a  detailed  description  is  available  in  [5]. 
Monitors  were  NTSC  standard, 19"  diagonal,  color  CRT's  (Proton  Model  600T).  Operators  wore 
eyeglasses  or  eyeglass  clip-ons  with  appropriately  oriented  polarizer  filters  to  maintain  visual 
channel  separation.  Special  care  was  taken  to  ensure  that  filters  on  the  monitors  and  eyeglasses 
were  matched  to  provide  true  mono  or  stereo  (not  pseudostereo)  views  with  minimal  "ghosting". 
Contrast  of  the  taskboard  holes  with  their  immediate  surrounding  was  held  constant  throughout 
the  experiment.  The  mono  view  was  achieved  by  matching  the  axes  of  polarization  of  both 
eyeglass  filters  with  that  of  a  single  monitor.  For  the  direct  view  control  conditions  which  were  run 
during  two  of  the  experimental  test  sessions,  operators  were  required  to  wear  the  stereo  polarizer 
eyeglasses  or  clip-ons  to  provide  greater  comparability  with  TV  viewing  conditions.  For  all  TV 
Views,  distance  from  eye  to  screen  was  approximately  1.2  meters,  providing  a  horizontal  display 
field  of  view  of  approximately  17  degrees.  For  the  direct  view/remote  manipulation  control  session, 
eye  to  taskboard  center  distance  was  approximately  1.8  meters.  Approximate  values  are  reported 
since  operators'  head  movements  were  not  physically  constrained  during  testing. 

The  manipulator  used  ,  a  Central  Research  Laboratories  (CRL)  Model  G  mini-master/slave 
unit,  was  of  the  terminus  type,  where  only  the  end-point  forces  at  the  controller  handle  are  fed 
forward  from  and  back  to  the  operator.  Force  reflection  in  this  manipulator  is  achieved 
mechanically  by  means  of  antagonistic  cable  pairs  operating  each  of  the  3  degrees  of  freedom  in  the 
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"arm"  and  4  degrees  of  freedom  (counting  grip  open-close)  in  the  end-effector.  The  end-effector  was 
a  parallel  jaw  gripper.  A  gripper  lock  feature  of  the  manipulator  was  used  to  secure  the  grip  on  the 
peg  during  testing.  This  eliminated  the  operator  muscular  fatigue  that  would  result  from  having  to 
constantly  exert  a  pincer-type  force  to  hold  onto  the  peg  over  a  long  series  of  trials.  During  the 
direct  manipulation  control  session,  the  operator  sat  on  a  chair  in  front  of  the  taskboard  offset  to 
left  center  of  the  board  so  that  the  right  shoulder  was  aligned  to  the  taskboard  center.  Eye  to 
taskboard  center  distance  was  approximately  70  cm. 

The  PiH  taskboard  consisted  of  16  stainless-steel  holes  of  1  cm  diameter  and  2.5  cm  depth 
arranged  in  two  straight  intersecting  lines  of  8  holes  each  to  form  a  cross  shape.  Holes  in  the  arms  of 
the  cross  were  separated  from  their  nearest  neighbors  by  8  cm.  Across  the  gap  at  the  intersection  of 
the  vertical  and  horizontal  arms  of  the  cross,  the  holes  were  separated  by  16  cm.  Faces  of  the  holes 
were  flush  with  the  flat  surface  of  the  taskboard  and  covered  with  a  non-reflective  material.  A 
microswitch  was  positioned  at  the  bottom  of  each  hole  to  register  full  insertion  of  a  peg.  Two 
diameters  of  pegs  were  used:  .75  and  .94  cm.  The  entire  taskboard  assembly  was  attached  to  the  end 
flange  of  a  large  industrial  robot  arm  (Unimation  Puma  760).  Under  program  control,  the  robot  arm 
moved  the  taskboard  to  one  of  8  pre-defined  orientations  in  a  randomized  order  of  presentation.  The 
8  orientations  were  3°  and  5°  "off-normal"  in  the  up,  down,  left,  and  right  directions.  For  TV 
Viewing  sessions,  the  operator's  view  of  the  taskboard  was  blanked  out  during  the  period  when  the 
taskboard  was  repositioned  prior  to  each  test  trial.  Three  high-intensitv  incandescent  floodlights 
were  positioned  above,  to  the  right,  and  to  the  left  of  the  taskboard  to  reduce  contrast  of  the 
shadows  cast  on  the  taskboard  during  approach  to  the  holes  and  insertion  of  the  peg. 


Figure  1.  Viewing  Geometry.  Point  X  represents  the  vertical  and  horizontal  center  of  the  PiH 
taskboard.  Operators  were  in  position  A  while  controlling  the  manipulator.  For  the 
direct  view/direct  manipulation  test  session,  they  were  in  position  B. 
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3.3  Procedure 


Prior  to  experimental  data  collection,  operators  were  given  extensive  practice  (i.e.,  two 
separate  test  sessions  of  128  trials  each)  in  order  to  familiarize  themselves  with  the  manipulator, 
the  task,  and  the  test  procedures.  Each  operator  performed  one  practice  session  under  the  mono  TV 
viewing  condition  and  the  other  under  the  stereo  TV  viewing  condition.  Order  of  presentation  for 
mono  TV  viewing  and  stereo  TV  viewing  was  counterbalanced  between  2  groups  of  4  operators. 

For  the  experiment,  each  operator  was  required  to  perform  the  peg-in-hole  task  for  six 
test  sessions  of  128  trials  each.  Testing  sessions  typically  required  between  60  and  90  minutes  to 
complete.  All  6  testing  sessions  were  comprised  of  trials  that  measured  performance  under  2 
peg  movement  directions  (vertical  vs.  horizontal),  a  single  movement  distance  (24  cm),  2 
eccentricities  of  movement  (central  and  peripheral)  and  2  peg  diameters  (0.75,  and  0.94  cm). For 
the  first  64  trials  in  each  session  (including  practice  sessions),  the  .75  cm  diameter  peg  was  used 
followed  by  64  trials  in  which  the  .94  cm  peg  was  used.  Over  the  course  of  the  6  testing  sessions, 
all  operators  were  presented  with  2  TV  viewing  conditions  (i.e.,  stereo  and  mono),  as  well  as  2 
conditions  of  taskboard  orientation  (i.e.,  taskboard  fixed  and  moved  ).  In  addition,  2  control 
sessions  were  run.  In  one  of  these  sessions,  operators  viewed  the  taskboard  directly  (i.e.,  from 
Position  A  in  Fig.  1)  and  used  the  CRL  manipulator  to  perform  the  task.  In  the  other  control 
session,  operators  performed  the  task  under  direct  view  conditions  (i.e.,  from  Position  B  in  Fig. 

1)  using  their  arms  and  hands.  In  a  "taskboard  fixed"  session,  the  PiH  board  did  not  change  its 
orientation  for  the  entire  set  of  128  trials  administered.  In  the  fixed  orientation,  the  top  of  the 
taskboard  was  tilted  3°  off  the  vertical  axis  defined  by  the  "normal"  orientation.  For  the 
"taskboard  moved"  condition,  the  PiH  taskboard  was  changed  in  orientation  from  trial  to  trial. 

A  total  of  8  taskboard/manipulator  orientations  were  used.  The  movement  directions  and 
distances  were  the  same  for  all  operators,  however  the  order  of  presentation  was  randomized 
across  operators  and  sessions.  Thus,  the  total  set  of  768  trials  administered  during  the 
experiment  was  the  same  for  all  operators,  but  their  order  of  presentation  was  both  randomized 
and  counterbalanced  across  operators  and  sessions  to  control  for  learning  effects 

A  single  trial  was  initiated  when  the  operator  s  prompting  CRT  indicated  the  START  and 
END  holes  which  the  peg  was  to  be  moved  between.  Once  the  operator  placed  the  peg  into  the 
START  hole,  the  control  computer  counted-off  a  1 -second  delay,  then  illuminated  a  green  LED  at 
the  control  station.  This  1-second  delay  imposed  a  standard  minimal  rest  period  prior  to  peg 
movement  for  all  trials.  Following  illumination  of  the  green  LED,  the  operator  lifted  the  peg  out  of 
the  START  hole,  moved  it,  and  inserted  it  into  the  END  hole.  Elapsed  time  required  to  move  the 
peg  from  the  bottom  of  the  START  hole  to  the  bottom  of  the  END  hole  was  provided  as  immediate 
feedback  on  the  operators'  prompting  CRT  and  recorded  for  subsequent  analysis. 

4.  RESULTS 

4.1  Practice  Effects 


To  assess  the  impact  of  practice  effects  on  PiH  task  performance  over  a  large  number  of 
trials,  an  analysis  was  conducted  on  data  derived  from  the  two  practice  sessions.  Movement  times 
for  8  operators  were  averaged  for  each  of  the  256  practice  trials.  It  should  be  noted  that  this 
averaging  blurred  the  distinction  between  mono  and  stereo  TV  viewing  since  order  of  presentation  of 
these  2  conditions  was  counterbalanced  for  equal-sized  groups  of  operators.  Next,  a  separate  linear 
regression  line  was  fitted  to  each  block  of  64  trials,  since  different-tolerance  pegs  were  used  during 
these  blocks  of  trials.  For  each  of  the  4  regression  lines  thus  generated,  a  test  was  calculated  for  the 
hypothesis  that  the  slope  of  the  best-fit  line  was  zero  (i.e.,  level)  .  The  results  of  these  analyses 
arc  summarized  in  Figure  2.  Some  overall  improvement  in  performance  attributable  to  practice  was 
noted  over  the  first  3  blocks  of  trials.  The  slope  of  the  fourth  block  was  essentially  zero. 
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Figure  2.  Practice  effect  for  PiH  Task  performance.  Data  points  plotted  are  averaged  over 
multiple  operators  (N=8).  For  each  operator,  a  total  of  256  practice  trails  were 
administered  in  2  sessions.  A  regression  line  was  computed  for  each  peg  type  within 
each  session.  Its  slope  coefficient(fi)  and  the  probability  of  that  slope  being  zero  are 
reported  for  each  regression  line  in  the  figure. 

4.2  Stereo  TV  Viewing  Advantage 

Peg  movement  times  collected  over  the  total  of  54  individual  testing  sessions  were 
subjected  to  a  3-way  repeated-measures  analysis  of  variance  (ANOVA)  using  a  commercially 
available  statistics  package  [4]  to  determine  the  main  and  interactive  effects  of  viewing  condition 
(stereoscopic  or  monoscopic),  taskboard  orientation  (  fixed  or  moved),  and  peg  tolerance  (2.5  or  .6 
mm)  on  PiH  movement  times.  Though  the  effect  for  taskboard  orientation  was  in  the  predicted 
direction  of  slower  movement  times  for  "taskboard  moved"  sessions,  it  was  not  found  to  be 
statistically  significant,  nor  were  any  of  the  interaction  terms  in  the  3  factor  analysis.  The  lack  of 
a  significant  effect  for  taskboard  orientation  may  well  have  been  due  to  the  use  of  relatively 
slight  deviations  (i.e.,  3°  and  5°)  from  the  "normal"  orientation.  A  subsequent  experiment 
undertaken  in  our  lab,  but  not  detailed  here,  used  larger  deviations  (i.e.,  10°  and  20°  )  and  produced 
the  expected  result  of  a  stronger  and  statistically  significant  effect  for  taskboard  orientation.  For 
present  purposes,  however,  since  no  systematic  effect  for  taskboard  orientation  on  movement  times 
was  found,  the  factor  was  eliminated  from  further  analysis.  A  subsequent  2-way  repeated- 
measures  ANOVA  revealed  that  both  viewing  condition  and  peg  tolerance  exerted  statistically 
significant  effects  on  movement  times  in  the  predicted  directions.  The  stereo  TV  advantage  is 
affirmed  in  the  viewing  condition  main  effect  of  Table  1  (F  =  15.161,  p  =  .0046,  df  =  1)  and  in  Figure 
3.  Average  task  time  for  stereo  TV  viewing  conditions  provided  a  modest  (i.e.,  7%)  improvement 
in  time  required  to  complete  the  task,  but  the  effect  was  highly  significant.  The  main  effect  for 
peg  tolerance,  though  not  depicted  in  a  figure  here,  was  in  the  expected  direction  (i.e.,  the  higher 
tolerance  increased  movement  times  by  21%)  and  highly  significant  (F  =  94.518,  p  =  .0001,  df  =  1). 
No  significant  interaction  was  found  between  the  Viewing  Condition  and  Peg  Tolerance  factors 
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To  provide  a  comparison  of  PiH  performance  under  stereo  TV  vs.  direct  view  conditions,  a  2- 
wav  ANOVA  was  run  on  data  collected  during  2  sessions  for  each  operator:  1)  stereo  TV  view, 
manipulator  used,  with  the  taskboard  moved  from  trial-to-trial,  and  2)  direct  view,  manipulator 
used,  with  the  taskboard  moved  from  trial-to-trial.  In  both  cases,  the  operator  was  situated  at 
Position  A  in  Fig.  l.The  two  factors  included  in  the  ANOVA  were  Viewing  Condition  (  direct  view 
vs  stereo  TV)  and  Peg  Tolerance.  The  main  effect  for  Viewing  Condition  failed  to  reach  statistical 
significance  (  F  =  0.21,  p  =  .66,  df  =  1) ,  but  the  Peg  Tolerance  effect  was  once  again  found  to  be  highly 
significant  (F  =  29.76,  p  =  .0006,  df  =  1).  More  importantly,  however,  the  interaction  of  Peg 
Tolerance  with  Viewing  Condition  was  significant  (F  =  18.68  ,  p  -  .002 5,  df  =  1  ).  A  plot  of  the  cell 
means  for  this  effect  is  found  in  Figure  4.  The  stereo  TV  viewing  condition  was  found  to  yield  faster 
performance  times  than  the  direct  viewing  condition  when  peg  tolerance  was  low,  and  slower 


performance  times  when  peg  tolerance  was  high.  In  interpreting  this  finding,  however,  it  must  be 
remembered,  that  the  angle  of  regard  differed  for  the  2  views  (see  Figure  1)  in  addition  to  the 
other  differences  between  TV  and  direct  view  conditions. 


Count  Mean  Std.  Dev.  Std.  Error 

Stereo  TV  View,  Dia.  =  .75  CM 
Stereo  TV  View,  Dia.  =  .94  CM 
Direct  View,  Dia.  =  .75  CM 
Direct  View,  Dia.  =  .94  CM 
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Figure  4.  Plot  of  PiH  Task  Performance  with  the  CRL  Manipulator  Comparing 
Direct  View  with  Stereo  TV  View. 


4.4  PiH  Performance  with  Manipulator  Versus  Human  Hand 

To  provide  a  comparison  of  PiH  performance  with  the  manipulator  vs.  human  hand,  a  2- 
way  ANOVA  was  run  on  data  collected  from  2  sessions  for  each  operator:  1 )  direct  view,  human  arm 
and  hand  used,  with  the  taskboard  moved  from  trial-to-trial,  and  2)  direct  view,  manipulator 
used,  with  the  taskboard  moved  from  trial-to-trial.  The  2  factors  included  in  the  ANOVA  were 
Manipulation  Type  (human  hand  and  CRL  manipulator)  and  Peg  Tolerance.  The  analysis  yielded 
highly  significant  main  effects  for  both  factors,  but  a  non-significant  interaction  (F  =  .36,  p  =  .56,  df 
=  1).  The  contrast  between  movement  times  with  the  human  hand  and  the  manipulator  (F  =  98.8,  p  < 
.0001,  df  =  1)  yielded  a  near  3-fold  increase  in  performance  times  in  going  from  hand  to  manipulator. 
This  effect  is  depicted  in  Figure  5.  The  Peg  Tolerance  effect  (  F  =  9.62,  p  =  .0146,  df  =  1 )  was  once 
again  strong  and  in  the  predicted  direction  of  slower  response  times  for  the  highf  tolerance. 
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53.016 

Figure  5.  Plot  of  PiH  Task  Performance  Comparing  Manipulator  and  Human 
Hand  Under  Direct  Viewing  Conditions 


5.  CONCLUSIONS 

Even  though  the  PiH  task  used  in  this  study  is  sometimes  regarded  as  one  of  the  least 
demanding  and  easiest  of  manipulation  tasks  to  master  with  minimal  prior  training,  the  results  of 
the  experiment  reported  here  demonstrate  that  such  "conventional  wisdom"  may  be  at  odds  with 
objective  reality.  The  findings  suggest  that  practice  effects  can  strongly  influence  performance 
outcomes  for  the  PiH  task  over  the  course  of  more  trials  than  are  frequently  given  for  practice  prior 
to  data  collection  when  comparisons  of  remote  manipulation  with  alternate  hardware  systems  are 
made.  Given  adequate  controls  for  carry-over  effects  (between-operator  designs,  randomization 
and/or  counterbalancing  for  orders  of  presentation),  the  effects  of  practice  that  go  on  within  and 
across  test  sessions  may  not  necessarily  contaminate  or  invalidate  experimental  findings,  but  they 
almost  certainly  will  add  unaccounted-for  variability  to  the  data  collected,  and  this  will  decrease 
the  likelihood  of  showing  a  statistically  significant  difference  between  equipment  features  when 
such  differences  do,  in  fact,  exist.  Although  the  practice  effects  reported  here  are  in  the  strictest 
sense  described  only  for  the  PiH  task,  they  suggest  that  "  more  complex"  tasks  may  require  a 
similar  amount  or  even  more  practice  by  operators  in  order  to  reach  a  desired  level  of  stabile 
performance  prior  to  comparison  testing. 

In  several  important  aspects  of  implementation  (i.e.,  image  resolution,  contrast,  channel 
registration  and  crosstalk,  system  magnification),  the  precision  and  quality  of  the  stereo  TV 
display  used  in  this  study  falls  somewhat  short  of  the  current  state-of-the-art  for  stereo  TV 
displays  exemplified  by  other  presentations  on  new  hardware  systems  given  at  this  conference. 
However,  the  main  purpose-  of  the  effort  described  here  was  not  to  assess  the  relative  merits  of  the 
particular  stereo  TV  display  or  the  manipulator  used.  Rather,  it  was  to  refine  test  methods  in  order 
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to  make  more  precise  measures  of  the  impact  of  system  features  on  overall  remote  manipulator 
system  performance.  Given  a  sensitive,  unbiased  testing  methodology,  a  practiced  pool  of 
operators,  a  state-of-the  -art  force-feedback  manipulator,  a  clear-cut  task,  diffuse  illumination  of 
the  taskboard,  and  an  "adequate"  stereo  TV  viewing  system  ,  it  was  possible  to  show  a  highly 
significant  performance  advantage  for  stereo  TV  over  a  directly  comparable  mono  TV  viewing 
system.  Though  the  demonstrated  PiH  speed  advantage  for  stereo  W  is  less  than  that  shown  by  a 
previous  experiment  at  NOSC  using  a  similar  stereo  TV  display  (9)  ,  the  main  findings  of  that 
earlier  experiment  were  replicated  and  confirmed  here.  Movement  times,  in  general,  were  roughly  3 
to  4  times  longer  for  the  earlier  study.  Some  of  this  discrepancy  can  be  attributed  to  the  practice 
operators  received  as  well  as  the  complexity  of  the  PiH  task  used.  The  previous  study  required 
operators  to  grasp  the  peg  prior  to  moving  it  from  one  hole  to  another.  Improvements  in  stereo  TV 
equipment  can  reasonably  be  expected  to  yield  improvements  in  measured  performance  of  tasks  like 
the  one  employed  here,  but  to  reveal  these  differences  as  statistically  significant  effects  will  be 
increasingly  difficult  without  a  test  method  that  is  sensitive  to  differences  when  they  occur. 

The  finding  of  no  significant  effect  for  taskboard  orientation  on  PiH  task  performance  was 
unexpected  given  the  assumption  that ,  other  factors  being  equal,  the  more  predictable  a  task,  the 
more  efficiently  it  should  be  performed.  Lack  of  a  significant  interaction  between  taskboard 
orientation  and  TV  viewing  condition  was  also  surprising  due  to  the  expectation  that  stereo 
viewing  would  provide  the  operator  with  a  more  accurate  perception  of  the  slant  of  the  taskboard 
and  that  this  would,  in  turn,  allow  him/her  to  make  more  accurate  gross  positioning  and  precision 
insertion  movements.  One  possibility  for  the  lack  of  any  significant  findings  is  suggested  by  data 
from  a  follow-on  (as  yet  unpublished)  experiment.  When  taskboard  orientation  was  varied  in  a 
more  extreme  manner  (i.e.,  to  10°  and  20°  deviations  off-normal),  a  modest,  but  statistically 
significant  main  effect  was  shown  in  the  predicted  direction,  but  no  interaction  with  viewing 
condition  was  found.  Other  explanations  can  only  be  speculated  about  at  present  due  to  a  lack  of 
performance  data.  Another  possibility,  based  on  the  observation  that  one  good,  clear  view  or 
physical  contact  with  the  board  might  be  sufficient  to  provide  the  operator  with  an  accurate 
impression  of  its  position  and  orientation  is  suggested  by  Merritt  [10].  By  inserting  the  peg  into  the 
START  hole  ,  the  operator  would  gain  considerable  information  about  the  position  and  orientation 
of  the  END  hole  by  correctly  assuming  that  the  surface  of  the  taskboard  was  flat.  Throughout  the 
entire  experiment  detailed  here,  the  averaged  depth  of  the  board  was  unchanged  from  trial-to- 
trial.  Only  its  orientation,  or  slant  angle  changed,  while  the  position  of  its  center  point  remained 
fixed  at  a  constant  depth  from  the  cameras.  Further  studies  are  underway  at  NOSC  to  characterize 
the  effects  of  positional  uncertainty  on  manipulator  performance. 

Results  from  the  comparison  between  performance  under  direct  vs  stereo  TV  viewing 
conditions  suggest,  but  do  not  conclusively  prove,  that  black-and-white,  NTSC  standard  stereo  TV 
does,  in  some  circumstances,  produce  performance  that  is  superior  to  that  achievable  under 
comparable  direct  viewing  conditions.  Moreover,  results  also  suggest  that  the  vantage  point  from 
which  a  task  is  attempted  may  be  equally,  or  even  more  important  than  some  aspects  of  the 
viewing  system  that  are  considered  to  be  critical  to  system  performance. 

Finally,  the  results  of  the  comparison  of  performance  between  the  human  hand  and  the 
manipulator  again  document  the  considerable  gap  that  separates  manipulator  performance  from 
direct  performance  of  the  task  -  even  when  viewing  conditions  are  "equally-natural",  though  not 
directly  comparable  with  respect  to  vantage  point. 
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